Bilateral Teleoperation System Research Articles

The nonlinear model reference adaptive impedance control for underwater manipulator operation objects in bilateral teleoperation system

The bilateral teleoperation system is susceptible to model parameter uncertainty and unknown disturbances in both the master and slave manipulators, resulting in instability and inaccuracies in the force and position tracking performance. To address these issues, a novel nonlinear model reference adaptive impedance controller has been designed to achieve coordinated force and position synchronization of dual manipulators. The adaptive control laws, based on sliding mode functions, have been designed to compensate for the uncertainty of the manipulator model. Furthermore, an adaptive estimation law has been employed to appraise the unknown upper bound of external disturbances. This ensures that the closed-loop model parameters of the dual manipulator converge to the reference impedance model respectively. Simultaneously, it enables the position error between the reference model response and the end-effector task space position to asymptotically converge to zero. To verify the effectiveness of the proposed controller, simulations have been conducted on the MATLAB platform and experiments on a single degree of freedom teleoperation system have been performed. The results demonstrate that the controller exhibits strong robustness and has the capability of force-position tracking ability.

Global terminal sliding‐mode control for the synchronization problem of uncertain bilateral teleoperation system with time‐varying delays

AbstractIn this paper, an adaptive global terminal sliding‐mode control is developed for a nonlinear bilateral teleoperation system with time‐varying delays, external disturbances, and uncertainties. By selecting a time‐varying sliding‐mode manifold, a finite‐time sliding‐mode control algorithm is proposed to guarantee the suppression of the reaching phase associated with the conventional sliding‐mode control. This implies that the robustness is guaranteed throughout to entire response of the system. Meanwhile, the chattering phenomenon and the energy consumption are significantly reduced. Furthermore, in order to enhance the tracking performance in practical situations, adaptive tuning laws are developed to estimate the unknown upper bounds of the lumped uncertainties. Hence, the proposed approach has several merits, namely, fast response, accurate synchronization, and strong robustness with respect to large uncertainties and disturbances. The stability of the system in closed‐loop and the boundedness on the internal dynamics are proved by Lyapunov stability theory. Illustrative simulations were conducted to show the effectiveness of the proposed controller.

Practical Approaches Towards Transparent and Stable Bilateral Teleoperation Under Time-Varying Network Delay

AbstractIn real-life teleoperation scenarios, the presence of time-varying network delays, particularly in wireless networks, poses significant challenges in maintaining stability in a bilateral teleoperation system. Various approaches have been proposed in the past to address stability concerns; however, these often come at the expense of system transparency. Nevertheless, increasing transparency is crucial in a teleoperation system to enable precise and safe operations, as well as to provide real-time decision-making capabilities for the operator. This paper presents our comprehensive approaches to maximize teleoperation transparency by minimizing system impedance, enhance the wave variable method to handle time-varying network delays, and alleviate non-smooth effects caused by network jitters in bilateral teleoperation. The proposed methodologies take into account the real-world challenges and considerations to ensure the practical applicability and effectiveness of the teleoperation system. Throughout these implementations, passivity analysis is employed to ensure system stability, guaranteeing a reliable and safe teleoperation experience. The proposed approaches were successfully validated in Team Northeastern’s Avatar telepresence system, which achieved the 3rd place in ANA Avatar XPRIZE challenge.

Robust passivity-based nonlinear controller design for bilateral teleoperation system under variable time delay and variable load disturbance

Abstract This research focuses on implementing a robust passivity-based nonlinear control method for bilateral/teleoperation systems. The key challenge is addressing communication pathways between the master and slave, control delays, and load disturbances, which can lead to instability and reduced transparency. To tackle these issues, the proposed controller incorporates a second-order super-twisting sliding-mode observer to counteract communication and control delays. A sliding mode assist disturbance observer compensates for load torque variations. The approach aims to ensure stability and transparency by handling time-varying delays. The system model comprises two interconnected direct-drive motors, simulating robotic configurations without a physical robot. The nonlinear controller framework simplifies the complex bilateral control problem, significantly improving stability and transparency performance. Computer simulations with step and sinusoidal inputs demonstrate the effectiveness of the approach, providing a satisfactory level of accuracy and transparency between estimated and actual slave positions, even with varying delays and load variations. The research contributes to control engineering by offering a robust method to enhance bilateral system performance, ensuring stable and transparent communication between the master and slave, particularly suitable for real-time internet-based bilateral control systems.

Position and Force Control of Bilateral Teleoperation Systems With Time-varying Delays Based on Force Estimation

Position and Force Control of Bilateral Teleoperation Systems With Time-varying Delays Based on Force Estimation

A USDE-Based Control for Bilateral Teleoperation Systems With Time-Varying Delays

A USDE-Based Control for Bilateral Teleoperation Systems With Time-Varying Delays

A Bilateral Teleoperation System With Learning-Based Cognitive Guiding Force

This article investigates the problem involving a fuzzy logic system-based bilateral teleoperation system with a unified impedance/admittance structure and learning-based assisted cognitive guiding force, enhancing a smooth transition between augmentation and autonomous motion on the master side. The impedance structure regulates the impedance and admittance via the parameters determined by the fuzzy logic system and, besides the variable impedance during interaction with the environment, the fuzzy Q-learning algorithm generates an assisted cognitive guiding force to guide the trainee operator in free space for reaching the interaction surface smoothly and steadily. The degeneration of tracking performance in the bilateral teleoperation system caused by time delay is analyzed and reduced via a mixed-type error. Experimental results verify the stability analyses, and under the proposed control scheme, the human-robot-environment interaction is smooth and stable.

Observer-based fuzzy integral sliding mode control for bilateral teleoperation systems with time-varying delays

This paper aims to examine the stability and tracking performance of a bilateral teleoperation system. The Takagi–Sugeno fuzzy method is utilized, through which the nonlinear master–slave dynamics is converted as a fuzzy-based system. The state observers are designed for the linearized fuzzy teleoperation systems, and the corresponding estimation errors are formulated. Importantly, a novel observer-based fuzzy integral sliding mode control is developed by deliberately introducing the delay term into the sliding surfaces. As such, advanced delay-product type of Lyapunov–Krasovskii functionals are constructed for the augmented state vectors, in order to acquire the additional delay information. In addition, the Wirtinger-based integral inequality along with an extended reciprocally convex matrix inequality is applied to the Lyapunov derivatives to establish the delay-dependent stability conditions. Numerical results are provided to demonstrate efficacy of the developed control mechanism.

The PID in the Control of Bilateral Teleoperation System with Time-Varying Delays: An Experimental Approach

The control of bilateral teleoperation systems with time-varying delays is a challenging problem that is usually dealt with advanced control techniques. The well-known controllers, such as PD or PID, are scarcely employed and usually in conjunction with other approaches or at least with gravity compensation. In this contribution we intend to show, experimentally, that the standard PID controller exhibits good performance and robustness in position control for teleoperators with variable delays. The results obtained are compared with a well-known algorithm used for teleoperator control.

Design and research of bilateral teleoperation for CFETR tile assembly using haptic feedback

Bilateral teleoperation is a control strategy used in remote handling (RH) to enable an operator to control a robot or system while perceiving the interactive force in the environment. To enhance the immersive operating experience by reducing time delays, a control system based on OROCOS and DDS has been presented to rapidly establish the bilateral teleoperation system with real-time performance. Additionally, a bilateral control algorithm combining closed-loop pose tracking and force feedback is proposed to improve the pose tracking accuracy of the slave robot and enhance the sense of telepresence. As a part of the pre-study on the tile assembly conducted by CMOR in Chinese Fusion Engineering Test Reactor (CFETR), a lightweight experimental setup has been constructed based on the maintenance scenario in EAST. Relative experiments have been conducted to validate the accuracy and effectiveness of the bilateral teleoperation system. The results demonstrate low latency and smooth control during the bilateral teleoperation process.

A Long Short-Term Memory-Based Model for Kinesthetic Data Reduction

This paper proposes a novel mathematical model for teleoperation over communication networks. For teleoperation over a communication network, a high packet rate can result in inefficient data transmission and cross-traffic problems, leading to extra delay and jitter. This paper proposes an LSTM-based mathematical model which focuses on kinesthetic data reduction without loss of transparency during the transmission process through joint training combined with haptic data and perceptual deadband. Since the LSTM network can deal with a time series of haptic data, we further test the system performance through practically collected data. We investigate the packet rate and perceptual transparency of the proposed mathematical model by comparing with the conventional deadband. Additionally, we compare the proposed mathematical model with the perceptual deadband-based codecs. Simulation results show that the proposed solution further reduces the packet rate when dealing with haptic data without noticeable distortion. Also, comparing with the current JND perceptual threshold, the proposed mathematical model helps improve the practicality of the bilateral teleoperation system without losing transparency.

Design of robust adaptive fuzzy control for uncertain bilateral teleoperation systems based on backstepping approach

AbstractIn this study, a novel method based on a robust adaptive fuzzy control approach is developed for nonlinear teleoperation systems. Its main objectives are to ensure system stability and properly mitigating parametric uncertainties stemming from external disturbances and un‐modelled dynamics. For the communication channel, instead of the direct transmission of environmental torque signals, the approximated environmental parameters by the fuzzy system are transmitted to the master side for the prediction of environmental torque, thus successfully avoiding the transmission of the power signals in the delayed communication channel and solving the passivity problem in the teleoperation system. Besides, a trajectory generator is employed in the master side, whereas a trajectory smoothing is provided in the slave side. Theoretically, it was proven that both position tracking and force feedback problems are attained. Using Lyapunov stability analysis, this work illustrates that the robust adaptive fuzzy controller based on the backstepping approach guarantees the system's asymptotic stability. Simulation results confirm the efficiency of the suggested control technique in achieving the stability and tracking objectives of the uncertain nonlinear teleoperation system.

Bilateral Teleoperation System with Integrated Position/Force Impedance Control for Assembly Tasks

This article investigates the realization of achieving safe and flexible assembly under manual teleoperation. A wearable positioning system for teleoperation assembly tasks was designed to provide great flexibility and operability. The 6D coordinate information of the hand was reconstructed with a wireless locator in real-time, and three control methods were conducted. In contrast to the traditional impedance methods, an integrated position/force control method which takes the operator’s posture as the desired position was proposed, thus achieving the combination of the initiative of the operator and the compliance of the impedance control. Additionally, the method possesses the capacity of eliminating collision force caused by hand jitters and misoperation. Finally, the system was evaluated in a representative application of teleoperated peg-in-hole insertion. Additionally, a challenging task was tested to illustrate advantages of the proposed method. The results show that the position trailing is precise enough for a teleoperation system, and the proposed integrated position/force control method approaches outperformed position control and impedance in terms of precision and operability.

Data-driven [formula omitted] control of constrained systems: An application to bilateral teleoperation system

A novel, identification-free, data-driven (DD) H∞ control method is presented for discrete-time (DT) linear time-invariant (LTI) systems under physical limitations and norm-bounded disturbances. The presented approach does not demand information on system matrices or any measurements of disturbance affecting the system. The only information needed to develop a static state-feedback (SF) controller is the bounds on disturbances, states and control signals. It is assumed that only the disturbance input matrix and the performance matrices the user generally defines are known, and all others are entirely unknown. The proposed method relies on the closed-loop (CL) parametrization of the LTI system with control input and state measurements. The disturbances affecting the system states are handled as affine uncertainties, later represented as Linear Fractional Transformation (LFT). For obtaining a less conservative controller, a full block S-procedure method (FBSPM) is used, which takes advantage of relaxations such as convex hull relaxation or Pólya relaxation for the inner approximation of the disturbance set with arbitrary precision. Numerical illustrations and extensive case studies on a bilateral teleoperation system indicate that the proposed design method allows us to obtain very effective controllers which never exceed the bounds of the state and input variables and are capable of reference and force tracking.

Position Measurement Based Slave Torque Feedback Control for Teleoperation Systems With Time-Varying Communication Delays

Bilateral teleoperation system is referred to as a promising technology to extend human actions and intelligence to manipulating objects remotely. For the tracking control of teleoperation systems, velocity measurements are necessary to provide feedback information. However, due to hardware technology and cost constraints, the velocity measurements are not always available. In addition, the time-varying communication delay makes it challenging to achieve tracking task. This paper provides a solution to the issue of real-time tracking for teleoperation systems, subjected to unavailable velocity signals and time-varying communication delays. In order to estimate the velocity information, immersion and invariance (I&I) technique is employed to develop an exponential stability velocity observer. For the proposed velocity observer, a linear relationship between position and observation state is constructed, through which the need of solving partial differential and certain integral equations can be avoided. Meanwhile, the mean value theorem is exploited to separate the observation error terms, and hence, all functions in our observer can be analytically expressed. With the estimated velocity information, a slave-torque feedback control law is presented. A novel Lyapunov-Krasovskii functional is constructed to establish asymptotic tracking conditions. In particular, the relationship between the controller design parameters and the allowable maximum delay values is provided. Finally, simulation and experimental results reveal that the proposed velocity observer and controller can guarantee that the observation errors and tracking error converge to zero.

H∞ control of a bilateral teleoperation system: Application to a PV system

This paper presents a novel approach for the H∞ control of a bilateral teleoperation system by combining dilated Linear Matrix Inequalities (LMIs) and the Takagi-Sugeno (T-S) fuzzy approach. The proposed methodology aims to enhance the control performance and robustness of bilateral teleoperation systems by leveraging the advantages of both dilated LMIs and T-S fuzzy modeling. The proposed approach ensures robust stability by utilizing Lyapunov stability analysis. Robust H∞ state feedback controllers are developed using both standard and dilated linear matrix inequalities to accommodate uncertain-ties inherent in the master/slave system. As an illustrative example, we enrich our approach by integrating a simulation model of a photovoltaic system. This extension demonstrates the versatility of our methodology by applying it to a distinct domain.

Event-triggered aperiodic intermittent sliding-mode control for master–slave bilateral teleoperation robotic systems

PurposeThe purpose of this paper is to propose a novel event-triggered aperiodic intermittent sliding-mode control (ETAI-SMC) algorithm for master–slave bilateral teleoperation robotic systems to further save communication resources while maintaining synchronization precision.Design/methodology/approachBy using the Lyapunov theory, a new event-triggered aperiodic intermittent sliding-mode controller is designed to synchronize master–slave robots in a discontinuous method. Unlike traditional periodic time-triggered continuous control strategy, a new ETAI condition is discussed for less communication pressure. Then, the exponential reaching law is adopted to accelerate sliding-mode variables convergence, which has a significant effect on synchronization performance. In addition, the authors use quantizers to make their algorithm have obvious progress in saving communication resources.FindingsThe proposed control algorithm performance is validated by an experiment developed on a practical bilateral teleoperation system with two PHANToM Omni robotic devices. As a result, the synchronization error is limited within a small range and the control frequency is evidently reduced. Compared with a conventional control algorithm, the experimental results illustrate that the proposed control algorithm is more sensitive to system states changes and it can further save communication resources while guaranteeing the system synchronization accuracy, which is more practical for real bilateral teleoperation robotic systems.Originality/valueA novel ETAI-SMC for bilateral teleoperation robotic systems is proposed to find a balance between reducing the control frequency and synchronization control precision. Combining the traditional sliding-mode control algorithm with the periodic intermittent control strategy and the event-triggered control strategy has produced obvious effect on our control performance. The proposed ETAI-SMC algorithm helps the controller be more sensitive to system states changes, which makes it possible to achieve precise control with lower control frequency. Moreover, we design an environment contact force feedback algorithm for operators to improve the perception of the slave robot working environment. In addition, quantizers and the exponential convergence law are adopted to help the proposed algorithm perform better in saving communication resources and improving synchronization precision.

Deep Learning-based Attack Detector for Bilateral Teleoperation Systems

A teleoperation system is referred to as a plant that is controlled remotely, and it is often composed of a human operator, a local master manipulator, and a remote slave manipulator, all connected by a communication network. Bilateral teleoperation systems (BTOS) include transmissions in both the forward and backward directions between the master and slave. This paper discusses a class of (BTOS) focusing on the security of the system after modeling the master and slave robots mathematically. The false data injection attack is examined, where the attacker may inject false data into the states that are being exchanged between the master and slave robots. The vulnerability of BTOS, where the attack destabilizes the system, is presented. A deep learning-based detection technique is proposed to detect the presence of false data injection attacks. The deep learning model with convolution neural network structure is trained and tested with considering complex attacks where the attacker has full knowledge of the system and proficiency to emanate and control the target system. The proposed model achieves 96\% validation accuracy, and the efficacy of the proposed deep learning detector is demonstrated and tested into the BTOS.

Adaptive control for position and force tracking of uncertain teleoperation with actuators saturation and asymmetric varying time delays

Abstract This paper presents a new bounded force feedback control law to improve transparency in nonlinear bilateral teleoperation systems in the presence of three problems in practical applications of teleoperation systems such as input saturation, asymmetric time varying communication delays with no restriction on their rates of variation and parametric uncertainties, simultaneously. The proposed controller is a nonlinear-proportional plus nonlinear damping (nP + nD) controller with the addition of a nonlinear adaptive term and nonlinear function of the environment force on the slave side and nonlinear function of the human force and force error on the master side. Using a novel Lyapunov–Krasovskii functional, the asymptotic stability and position and force tracking performance of the teleoperation system are established under specific conditions on the controller parameters, actuator saturation characteristics and maximum allowable time delay. The validity of the theoretical results is corroborated by the simulation results.

Design of Synchronization Tracking Adaptive Control for Bilateral Teleoperation System with Time-Varying Delays.

The performances of position synchronization and force interaction of the teleoperation system provide a safe and efficient way for operators to perform tasks in remote, hazardous environments. In practice, however, communication delays and dynamic uncertainties can impair the performance of position synchronization controls. Under the above factors, it is necessary to study and design appropriate bilateral control methods to achieve stable and effective position synchronization control. In this paper, a new adaptive control architecture based on velocity feedback filter and radial basis function neural network is proposed. In the proposed control scheme, only the position signal is transmitted during the communication process, and the speed feedback filter and compensation method are designed and adopted to avoid the use of acceleration signals. In addition, a new auxiliary variable with a tracking error integral term is used to reduce the steady-state error of position tracking under nonzero external environmental forces. Using the Lyapunov–Krasovskii method, the stability of closed-loop remote operating systems is demonstrated. In the simulation and experiment sections, the algorithm was verified separately and compared with other algorithms. The results of a master–slave robot system verify the tracking performance of our proposed control scheme.